Method and apparatus for processing information, storage medium, and program

ABSTRACT

In a television set, image quality is adjusted in accordance with parameters calculated on the basis of adjustment values used in the past, depending on a feature of an image and an environmental status. A weight calculator determines a weight depending on the manner in which a user performs a volume control operation. A cumulative weight memory outputs a cumulative weight corresponding to the feature value. Output volume values used in past are stored in a volume value generator. The volume value generator calculates output volume values corresponding to a feature value on the basis of parameters indicating final adjustment values, the feature value, the weight, and the cumulative weight. The calculated volume values are stored in the volume value generator. Under the control of a system controller, the volume values stored in the volume value generator are output to a coefficient generator.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a method and apparatus forprocessing information, a recording medium, and a program, and moreparticularly, to a method and apparatus for processing information, arecording medium, and a program, which allow a user to make anadjustment associated with content data such as an image in accordancewith user's preference.

[0003] 2. Description of the Related Art

[0004] In recent years, there has been a need for a high-performanceaudio/visual system. In response to such a need, a high-performancetelevision system called “high vision” (trademark) having the capabilityof displaying a higher-resolution image than can be obtained inconventional television sets have been developed. In the high visiontelevision system, in contrast to the NTSC television system in which525 scanning lines are used, as many as 1125 scanning lines are used.That is, the number of scanning lines used in the high vision televisionsystem is greater than twice the number of scanning lines used in theNTSC television system. Furthermore, in the high vision televisionsystem, unlike the NTSC television system in which the screen aspectratio is 3:4, the screen aspect ratio is 9:16, which makes it possibleto display a more realistic image with higher resolution than can beachieved by the NTSC television system.

[0005] Although the high vision television system has advantagesdescribed above, when a conventional NTSC video signal is directlysupplied to a high vision television set, the high vision television setcannot display it in the high vision format because of theabove-described differences in specifications between the NTSCtelevision system and the high vision television system.

[0006] In view of the above, the applicant for the present invention hasproposed (in Japanese Unexamined Patent Application Publication No.8-51599) a converter for converting a NTSC video signal into a highvision video signal thereby allowing an image corresponding to the NTSCvideo signal to be displayed in the high vision form. In this converter,for pixel data at a location of interest in a high vision video signal,pixel data in a block (area) at a location corresponding to the locationof interest in the high vision video signal are extracted from a givenNTSC video signal. A class of the pixel data at the location of interestis determined on the basis of a level distribution pattern of theextracted pixel data in the block. The specific value of the pixel dataat the location of interest is then determined in accordance with thedetermined class.

[0007] However, in the converter described above, the resultant imagedisplayed in the high vision format has a fixed resolution, and a usercannot adjust parameters such as contrast or sharpness associated withthe resolution, depending on the content of the image in accordance withuser's preference.

[0008] To avoid the above problem, the applicant for the presentinvention has further proposed (in Japanese Unexamined PatentApplication Publication No. 2001-238185) a NTSC-to-high vision videosignal conversion technique in which a high vision video signal isgenerated in accordance with values of parameters specified by a userthereby allowing the user to adjust the resolution of an image displayedin accordance with the resultant high vision video signal.

[0009] Although this video signal conversion technique allows a user toadjust the resolution of the image in accordance with user's preference,the technique has a further problem. For example, in a case in which adesirable resolution varies depending on whether an image has a brightcolor tone or dark color tone, the user has to adjust the resolutioneach time the color tone varies. This is very troublesome to the user.Furthermore, in a case in which there are two or more parametersassociated with image quality, if one parameter is adjusted, theadjustment of the one parameter can influence other parameters, and thusit is needed to readjust other parameters which have already beenadjusted. Thus, it is difficult to quickly achieve a desired result inthe adjustment.

SUMMARY OF THE INVENTION

[0010] In view of the above, an object of the present invention is toprovide a technique which allows a user to easily and quickly make anadjustment associated with an image.

[0011] To achieve the above object, the present invention provides afirst information processing apparatus comprising processing means forprocessing content data, acquisition means for acquiring firstinformation for controlling the processing means, and generation meansfor generating second information using a value obtained by weightingthe first information acquired by the acquisition means, wherein theprocessing means processes the content data on the basis of the secondinformation generated by the generation means.

[0012] The first information processing apparatus may further compriseinput means for receiving a command/data issued by a user, wherein theacquisition means may acquire, as the first information, an adjustmentvalue input by the user via the input means, and the processing meansprocess the content data such that when an automatic adjustment commandis input by the user via the input means, the processing means processesthe content data on the basis of the second information generated by thegeneration means, while in the case in which the automatic adjustmentcommand is not issued by the user via the input means, when theadjustment value is input by the user via the input means, theprocessing means processes the content data on the basis of the firstinformation acquired by the acquisition means.

[0013] The first information apparatus may further comprise featuredetection means for detecting features of the content data, wherein thegeneration means may generate second information for each featuredetected by the feature detection means for the content data, and theprocessing means may process the content data using the secondinformation corresponding to the feature of the content data detected bythe feature detection means.

[0014] The feature detection means may detect, as a feature of thecontent data, the variance of image levels.

[0015] The feature detection means may detect, as a feature of thecontent data, the mean image level.

[0016] The first information processing apparatus may further compriseenvironmental information detection means for detecting environmentalinformation associated with an environmental condition, wherein thegeneration means may generate second information for each piece ofenvironmental information detected by the environmental informationdetection means, and the processing means may process the content datausing second information corresponding to the environmental informationdetected by the environmental information detection means.

[0017] The environmental information detection means may detect, as theenvironmental information, the temperature in the ambient.

[0018] The environmental information detection means may detect, as theenvironmental information, the humidity in the ambient.

[0019] The environmental information detection means may detect, as theenvironmental information, the brightness of a light in the ambient.

[0020] The first information processing apparatus may further compriseinformation extraction means for extracting information associated withthe content data, wherein the generation means may generate secondinformation for each piece of information extracted by the informationextraction means, and the processing means may process the content datausing second information corresponding to the information extracted bythe information extraction means.

[0021] The first information processing apparatus may further comprisestorage means for storing the second information generated by thegeneration means.

[0022] The storage means may be formed such that it can be removed fromthe information processing apparatus.

[0023] The present invention also provides a first informationprocessing method comprising the steps of processing the content data,acquiring first information for controlling the processing step, andgenerating second information using a value obtained by weighting thefirst information acquired in the acquisition step, wherein in theprocessing step, the content data is processed on the basis of thesecond information generated in the generation step.

[0024] The present invention also provides a first storage mediumincluding a program stored thereon comprising the steps of processingthe content data, acquiring first information for controlling theprocessing step, and generating second information using a valueobtained by weighting the first information acquired in the acquisitionstep, wherein in the processing step, the content data is processed onthe basis of the second information generated in the generation step.

[0025] The present invention also provides a first program comprisingthe steps of processing the content data, acquiring first informationfor controlling the processing step, and generating second informationusing a value obtained by weighting the first information acquired inthe acquisition step, wherein in the processing step, the content datais processed on the basis of the second information generated in thegeneration step.

[0026] The present invention also provides a second informationprocessing apparatus comprising processing means for processing contentdata, acquisition means for acquiring first information and secondinformation for controlling the processing means, detection means fordetecting a relationship between the first information and the secondinformation acquired by the acquisition means, and generation means forgenerating third information and fourth information by converting thefirst information and the second information in accordance with therelationship detected by the detection means, wherein the processingmeans processes the content data in accordance with the relationshipdetected by the detection means and the third information and fourthinformation generated by the generation means.

[0027] The detection means may detect the relationship between the firstinformation and the second information, by using a linear expression.

[0028] The detection means may detect the relationship between the firstinformation and the second information, by using a high-orderexpression.

[0029] The detection means may detect the relationship between the firstinformation and the second information, by using a vector quantizationtable and vector quantization codes.

[0030] The detection means may calculate coordinate axes on the basis ofthe detected relationship between the first information and the secondinformation, and the detection means may produce a conversion table usedto generate the third information and the fourth information byconverting the first information and the second information,respectively. The generation means may generate the third informationand the fourth information by converting the first information and thesecond information on the basis of the conversion table generated by thedetection means.

[0031] The second information processing apparatus may further comprisedisplay control means for controlling displaying of information otherthan the content data, wherein the display control means may controldisplaying of coordinates of the third information and the fourthinformation generated by the generation means along the coordinate axescalculated by the detection means.

[0032] The second information processing apparatus may further comprisestorage means for storing the conversion table generated by thedetection means.

[0033] The second information processing apparatus may further comprisestorage means for storing the third information and the fourthinformation generated by the generation means.

[0034] The detection means may detect the relationship between the firstand the second information, when a greater number of pieces of thirdinformation and fourth information than a predetermined number arestored in the storage means.

[0035] The storage means may be formed such that it can be removed fromthe information processing apparatus.

[0036] The second information processing apparatus may further compriseinput means for receiving a command/data issued by a user, wherein thedetection means may detect the relationship between the firstinformation and the second information in response to receiving acommand to produce new coordinate axes issued by a user.

[0037] The present invention also provides a second informationprocessing method comprising the steps of processing the content data,acquiring first information and second information for controlling theprocessing step, detecting a relationship between the first informationand the second information acquired in the acquisition step, andgenerating third information and fourth information by converting thefirst information and the second information in accordance with therelationship detected in the detection step, wherein in the processingstep, the content data is processed in accordance with the relationshipdetected in the detection step and the third information and fourthinformation generated in the generation step.

[0038] The present invention also provides a second storage mediumincluding a program stored thereon comprising the steps of processingthe content data, acquiring first information and second information forcontrolling the processing step, detecting a relationship between thefirst information and the second information acquired in the acquisitionstep, and generating third information and fourth information byconverting the first information and the second information inaccordance with the relationship detected in the detection step, whereinin the processing step, the content data is processed in accordance withthe relationship detected in the detection step and the thirdinformation and fourth information generated in the generation step.

[0039] The present invention also provides a second program comprisingthe steps of processing the content data, acquiring first informationand second information for controlling the processing step, detecting arelationship between the first information and the second informationacquired in the acquisition step, and generating third information andfourth information by converting the first information and the secondinformation in accordance with the relationship detected in thedetection step, wherein in the processing step, the content data isprocessed in accordance with the relationship detected in the detectionstep and the third information and fourth information generated in thegeneration step.

[0040] In the first information processing apparatus, the firstinformation processing method, and the first program, content data isprocessed in such a manner that first information for controlling theprocessing means is acquired, second information is generated using avalue obtained by weighting the first information acquired by theacquisition means, and the content data is processed on the basis of thegenerated second information.

[0041] In the second information processing apparatus, the secondinformation processing method, and the second program, content data isprocessed in such a manner that first information and second informationfor controlling the processing means are acquired, a relationshipbetween the first information and the second information is detected,third information and fourth information are generated by converting thefirst information and the second information in accordance with thedetected relationship, and the content data is processed in accordancewith the detected relationship and the generated third information andfourth information.

BRIEF DESCRIPTION OF THE DRAWINGS

[0042]FIG. 1 is a block diagram showing the construction of a televisionset according to the present invention;

[0043]FIG. 2 is a diagram showing locations of pixels of a 525i signaland those of a 1050i signal;

[0044]FIG. 3 is a diagram showing phase differences, relative to acentral prediction tap, of four pixels in a unit pixel block of a HDsignal (1050i signal) in an odd field;

[0045]FIG. 4 is a diagram showing phase differences, relative to acentral prediction tap, of four pixels in a unit pixel block of a HDsignal (1050i signal) in an even field;

[0046]FIG. 5 is a block diagram showing the construction of a historyinformation memory;

[0047]FIG. 6 is a diagram showing an example of a user interface foradjusting image quality;

[0048]FIG. 7 is a diagram showing, in an enlarged fashion, a part of anadjustment screen shown in FIG. 6;

[0049]FIG. 8 is a flow chart showing a process changing volume values;

[0050]FIG. 9 is a diagram showing a simple mean value of stored volumevalues;

[0051]FIG. 10 is a diagram showing a weighted mean value of storedvolume values;

[0052]FIG. 11 is a diagram showing an example of a method of generatingcoefficient seed data;

[0053]FIG. 12 is a block diagram showing an example of a construction ofcoefficient seed data generator;

[0054]FIG. 13 is a diagram showing a change in range within whichresolutions are adjusted;

[0055]FIG. 14 is a block diagram showing another construction oftelevision set according to the present invention;

[0056]FIG. 15 is a block diagram showing the details of a historyinformation memory shown in FIG. 14;

[0057]FIG. 16 is a flow chart showing a process of changing volume axes;

[0058]FIG. 17 is a diagram showing a change of volume axes;

[0059]FIG. 18 is a diagram showing a change of volume axes;

[0060]FIG. 19 is a diagram showing a change of volume axes; and

[0061]FIG. 20 is a diagram showing a change of volume axes.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0062] The present invention is described in further detail below withreference to preferred embodiments in conjunction with the accompanyingdrawings.

[0063]FIG. 1 is a block diagram showing the construction of a televisionset 1. When the television set 1 acquires a 525i SD (StandardDefinition) signal from a broadcast signal, the television set 1converts the acquired 525i signal into a 1050i HD (High Definition)signal and displays an image in accordance with the HD signal. Herein,numerals of “515i” and “1050i” each denote the number of scanning lines,and “i” denotes “interlace”.

[0064]FIG. 2 shows locations of pixels in a certain frame (F) for both a525i signal and a 1050i signal. In FIG. 2, solid lines represent pixellocations in an odd (o) field, and dashed lines represent pixellocations in an even (e) field. Pixels of the 525i signal are indicatedby greater dots, and pixel of the 1050i signal are indicated by smallerdots. As can be seen from FIG. 2, pixel data of the 1050i signalincludes line data such as L1 and L1′ located close to 525i signal linesand line data such as L2 and L2′ located less close to 525i signallines, wherein L1 and L2 indicate typical line data in the odd field,and L1′ and L2′ indicate typical line data in the even field. Each lineof the 1050i signal includes as many pixels as twice the number ofpixels in each line of the 525i signal.

[0065] Referring again to FIG. 1, the construction of the television set1 is further described. A user operates the television set 1 using aremote commander 2. The television set 1 includes a system controller 12for controlling operations over the entire system, and a signal receiver11 for receiving a remote control signal. The system controller 12includes a microcontroller including a CPU (Central Processing Unit), aRAM (Random Access Memory), and a ROM (Read Only Memory). The signalreceiver 11 is connected to the system controller 12 such that when aremote control signal is output from the remote commander 2 in responseto a control operation performed by a user, the signal receiver 11receives the remote control signal and supplies a control signalcorresponding to the received remote control signal to the systemcontroller 12 .

[0066] The remote commander 2 has buttons used to control the operationof the television set 1. They include, for example, channel selectionbuttons, and audio volume control buttons. The remote commander 2 alsohas a joystick 81 for inputting image adjustment commands which will bedescribed later with reference to FIGS. 6 and 7, and has an automaticadjustment button 82 for automatically adjusting image quality bycalculating suitable adjustment values depending on the feature of animage or the environmental conditions and on the basis of adjustmentvalues employed in the past.

[0067] The system controller 12 is responsible for controlling theoperation over the entire television set 1. For the above purpose, thesystem controller 12 generates control signals and supplies them tovarious parts of the television set 1 to control them. The systemcontroller 12 is connected to a sensor 20 for detecting environmentalparameters indicating, for example, temperature, humidity, brightness ofa light, and the like in the ambient in which the television set 1 islocated. The environmental parameters acquired via the sensor 20 areoutput from the system controller 12 to a feature value extractor 56.Herein, a plurality of sensors 20 may be used.

[0068] A receiving antenna 3 receives a broadcast signal (modulated RFsignal). The broadcast signal captured by the receiving antenna 3 issupplied to a tuner 13. In accordance with a control signal supplied viathe system controller 12, the tuner 13 selects a channel specified by auser using the remote commander 2. In the tuner 13, intermediatefrequency amplification and detection are performed on the selectedchannel to obtain the SD signal (525i signal). A buffer memory 14temporarily stores the SD signal output from the tuner 13.

[0069] In a case in which the television set 1 has the capability ofreceiving digital broadcast data including a program table in a formataccording to EPG (Electronic Program Guide), the system controller 12extracts EPG data from received digital broadcast data (temporarilystored in the buffer memory 14) thereby acquiring the EPG data. Theprogram data includes information about broadcast programs of few days,in terms of, for example, broadcast date and time, a channel, a title,actor/actress's names, a genre, and an abstract.

[0070] The television set 1 includes, or is connectable to, arecording/playing back apparatus for recording/reproducing content dataonto/from recording medium such as an optical disk, a magnetic disk, amagnetooptical disk, a semiconductor memory, or a magnetic tape. In somecases, content data stored on a storage medium includes information,similar to EPG data, indicating a channel, a title, actor/actress'snames, a genre, and/or an abstract.

[0071] The system controller 12 detects information indicating a genreor an abstract of content of a program being viewed or recorded from EPGdata extracted from digital broadcast data or data, similar to EPG data,included in content data stored on a recording medium, and the systemcontroller 12 supplies the detected information to the feature valueextractor 56.

[0072] An image signal processing unit 15 is constructed such that theimage signal processing unit 15 or a circuit board including the imagesignal processing unit 15 can be removed from the television set 1 andcan be carried by a user. The image signal processing unit 15 performsimage signal processing to convert a SD signal (525i signal) temporarilystored in the buffer memory 14 into a HD signal (1050i signal).

[0073] In the image signal processing unit 15, a first tap selector 41,a second tap selector 42 and a third tap selector 43 selectively extractdata of a plurality of SD pixels in the vicinity of a location ofinterest in the HD signal (1050i signal) from the SD signal (525isignal) stored in the buffer memory 14, and the first to third tapselectors 41, 42 and 43 outputs the extracted data.

[0074] More specifically, the first tap selector 41 selectively extractsdata of SD pixels to be used for prediction (hereinafter, such SD pixelswill be referred to as “prediction taps”). The second tap selector 42selectively extracts data of SD pixels to be used for classification interms of the level distribution pattern of the SD pixel data(hereinafter referred to as (spatial class taps). The third tap selector43 selectively extracts data of SD pixels to be used for classificationin terms of motion (hereinafter, such SD pixels will be referred to as“motion class taps”). In a case in which a spatial class is determinedon the basis of data of SD pixels in a plurality of fields, the spatialclass also includes motion information.

[0075] A spatial class detector 44 detects the level distributionpattern of the spatial class tap data (SD pixel data) selectivelyextracted by the second tap selector 42. The spatial class detector 44then determines a spatial class on the basis of the detected leveldistribution pattern and outputs class information indicating thespatial class.

[0076] The spatial class detector 44 compresses each SD pixel data, forexample, from 8-bit data into 2-bit data and outputs, as spatial classinformation, the resultant compressed data corresponding to each SDpixel data. In the present embodiment, data compression is performedusing an ADRC (Adaptive Dynamic Range Coding) technique. Note that thetechnique of data compression is not limited to the ADRC but other datacompression techniques based on, for example, DPCM (predictive coding)or VQ (vector quantization) may also be employed.

[0077] The ADRC technique is a high-performance coding technique basedon adaptive requantization, which has been originally developed for usein VTRs (Video Tape Recorders). The ADRC technique can represent a localpattern of a signal level in an efficient manner using short-length-worddata, and thus the ADRC technique is suitable for use in the datacompression. In the case in which the ADRC technique is used, if themaximum and minimum values of the spatial class tap data (SD pixel data)are denoted by MAX and MIN, respectively, the dynamic range of thespatial class tap data by DR (=MAX−MIN+1), and the number of bits ofrequantized data by P, then a requantized code qi can be determined inthe form of compressed data for each SD pixel data ki of the spatialclass tap data, in accordance with equation (1). In equation (1), [ ]denotes a rounding-down operation. In a case in which the number of SDpixel data included in spatial class tap data is equal to Na, i inequation (1) takes values of 1, 2, 3, . . . , Na.

qi=[(ki−MIN+0.5)×2^(P) /DR]  (1)

[0078] The motion class detector 45 detects a motion class, chieflyindicating the degree of motion, from the data of motion class taps (SDpixel data) selectively extracted by the third tap selector 43, and themotion class detector 45 outputs class information indicating thedetected motion class.

[0079] More specifically, the motion class detector 45 calculatesinterframe differences from the motion class tap data (SD pixel data miand SD pixel data ni (i=1, 2, 3, . . . )) selectively extracted from thethird tap selector 43. Furthermore, the motion class detector 45determines the motion class, which is a measure of motion, bythresholding the mean value of absolute values of differences. In thisprocess, the motion class detector 45 calculates the mean value AV ofabsolute values of differences in accordance with equation (2). Forexample, in the case in which the third tap selector 43 extracts 12 SDpixel data m1 to m6 and n1 to n6 in the above-described manner, Nb inequation (2) is 6 (the maximum value of i). $\begin{matrix}{{AV} = \frac{\sum\limits_{i = 1}^{Nb}{{{m\quad i} - {ni}}}}{Nb}} & (2)\end{matrix}$

[0080] The motion class detector 45 compares the mean value AVcalculated in the above-described manner with one or more thresholdvalues, thereby acquiring class information MV indicating the motionclass. For example, in a case in which three threshold values th1, th2and th3 (th1<th2<th3) are prepared, motion is classified into one offour motion classes depending on the mean value AV such that MV=0 whenAV≦th1, MV=1 when th1<AV≦th2, MV=2 when th2<AV≦th3, or MV=3 when th3<AV.

[0081] On the basis of the requantized code qi output from the spatialclass detector 44 as the class information indicating the spatial classand the class information MV of indicating the motion class output fromthe motion class detector 45 the class combiner 46 determines a classcode CL indicating a class for pixel data (data of the pixel at thelocation of interest) of the HD signal (1050i signal) to be produced.

[0082] More specifically, the class combiner 46 calculates the classcode CL in accordance with equation (3). In equation (3), Na denotes thenumber of data of spatial class taps (SD pixel data), and P denotes thenumber of bits of quantized data generated in the ADRC process.$\begin{matrix}{{CL} = {{\sum\limits_{i = 1}^{Na}{{qi}\left( 2^{P} \right)}^{i - 1}} + {{MV} \cdot \left( 2^{P} \right)^{Na}}}} & (3)\end{matrix}$

[0083] A coefficient memory 53 stores a plurality of coefficient dataWi, for each class, used in the prediction equation used by a predictor47. The coefficient data Wi are information necessary in conversion froma SD signal (525i signal) to a HD signal (1050i signal). The class codeCL output from the class combiner 46 is supplied as read addressinformation to the coefficient memory 53. In response, the coefficientmemory 53 outputs coefficient data Wi (i=1 to n) which corresponds tothe class code CL and which is to be used in the prediction equation.The output coefficient data Wi are supplied to the predictor 47.

[0084] The image signal processing unit 15 includes an informationmemory bank 51. The predictor 47 calculates the HD pixel data y to begenerated, using data (SD pixel data) xi of prediction taps suppliedfrom the first tap selector 41 and the coefficient data Wi read from thecoefficient memory 53, in accordance with a prediction equation given byequation (4). In equation (4), n denotes the number of prediction tapsselected by the first tap selector 41.

[0085] Herein, n pixel data selectively extracted as the prediction tapsby the tap selector 41 are located in the spatial vicinity (inhorizontal and vertical directions) and in the temporal vicinity of thelocation of interest in the HD signal. $\begin{matrix}{y = {\sum\limits_{i = 1}^{n}{{Wi} \cdot {xi}}}} & (4)\end{matrix}$

[0086] The coefficient data Wi (i=1 to n) in the prediction equation aregenerated in accordance with a generation equation, given by equation(5), including parameters S and Z. The information memory bank 51 storescoefficient seed data w10-wn9 , for each class, which are used as thecoefficient data in the generation equation. The manner of generatingthe coefficient seed data will be described later. $\begin{matrix}\begin{matrix}\begin{matrix}{W_{1} = {w_{10} + {w_{11}s} + {w_{12}z} + {w_{13}s^{2}} + {w_{14}{sz}} + {w_{15}z^{2}} +}} \\{{{w_{16}s^{3}} + {w_{17}s^{2}z} + {w_{18}{sz}^{2}} + {w_{19}z^{3}}}}\end{matrix} \\\begin{matrix}{W_{2} = {w_{20} + {w_{21}s} + {w_{22}z} + {w_{23}s^{2}} + {w_{24}{sz}} + {w_{25}z^{2}} +}} \\{{{w_{26}s^{3}} + {w_{27}s^{2}z} + {w_{28}{sz}^{2}} + {w_{29}z^{3}}}}\end{matrix} \\{\quad \vdots} \\\begin{matrix}{W_{i} = {w_{i0} + {w_{i1}s} + {w_{i2}z} + {w_{i3}s^{2}} + {w_{i4}{sz}} + {w_{i5}z^{2}} +}} \\{{{w_{i6}s^{3}} + {w_{i7}s^{2}z} + {w_{i8}{sz}^{2}} + {w_{i9}z^{3}}}}\end{matrix} \\{\quad \vdots} \\\begin{matrix}{W_{n} = {w_{n0} + {w_{n1}s} + {w_{n2}z} + {w_{n3}s^{2}} + {w_{n4}{sz}} + {w_{n5}z^{2}} +}} \\{{{w_{n6}s^{3}} + {w_{n7}s^{2}z} + {w_{n8}{sz}^{2}} + {w_{n9}z^{3}}}}\end{matrix}\end{matrix} & (5)\end{matrix}$

[0087] As described above, in the conversion from a 525i signal into a1050i signal, it is required to obtain four pixels in the 1050i signalcorresponding to each pixel in the 525i signal in each odd field andalso in each even field, wherein 4 (=2×2) blocks in each unit pixelblock of the 1050i signal in each odd field and those in each even fieldare different in phase with respect to the central prediction tap fromeach other.

[0088]FIG. 3 shows phase differences, relative to the central predictiontap SD0, of four pixels HD1 to HD4 in a unit pixel block of a 1050isignal in an odd field. In this example, the locations of pixels HD1 toHD4 deviate by k1 to k4, respectively, from the location of the centralprediction tap SD0 in a horizontal direction, and by m1 to m4,respectively, in a vertical direction.

[0089]FIG. 4 shows phase differences, relative to the central predictiontap SD0′, of four pixels HD1′ to HD4′ in a 2×2 unit pixel block of a1050i signal in an even field. In this example, the locations of pixelsHD1′ to HD4′ deviate by k1′ to k4′, respectively, from the location ofthe central prediction tap SD0′ in the horizontal direction, and by m1′to m4′, respectively, in the vertical direction.

[0090] Thus, in the information memory bank 51, coefficient seed dataw10-wn9 are stored for each combination of a class and output pixels(HD1 to HD4 or HD1′ to HD4′).

[0091] For each class, a coefficient generator 52 generates coefficientdata Wi (i=1 to n) which correspond to the values of parameters S and Zand which are used in the prediction equation, using the coefficientseed data of each class loaded from the information memory bank 51 andthe values of parameters S and Z supplied from the system controller 12or a history information memory 50, in accordance with equation (5).

[0092] That is, the coefficient generator 52 receives not only thevalues of parameters S and Z from the system controller 12, but alsoreceive, instead of the parameters S and Z, volume values Sv and Zvcorresponding to the feature value extracted by the feature valueextractor 56 from the history information memory 50. In this case, thecoefficient generator 52 substitutes the volume values Sv and Zv,instead of the parameters S and Z, into equation (5) to generate thecoefficient data Wi (i=1 to n).

[0093] The coefficient data Wi (i=1 to n) of each class generated by thecoefficient generator 52 are stored in the coefficient memory 53described above. The generation of the coefficient data Wi (i=1 to n) ofeach class by the coefficient generator 52 is performed, for example, ineach vertical blanking period. Thus, when the values of parameters S andZ are changed in response to a control operation performed by a user onthe remote commander 2, the coefficient data Wi of each class stored inthe coefficient memory 53 are immediately changed to valuescorresponding to the new values of the parameters S and Z, therebyallowing the user to smoothly adjust the resolution.

[0094] A normalization coefficient calculator 54 calculates, inaccordance with equation (6), the normalization coefficients Sncorresponding to the coefficient data Wi (i=1 to n) determined by thecoefficient generator 52. $\begin{matrix}{S_{n} = {\sum\limits_{i = 1}^{n}W_{i}}} & (6)\end{matrix}$

[0095] The resultant calculated normalization coefficients Sn are storedin a normalization coefficient memory 55. The class code CL output fromthe class combiner 46 described above is supplied as read addressinformation to the normalization coefficient memory 55. In response, thenormalization coefficient memory 55 reads normalized coefficient Sncorresponding to the class code CL and supplies them to the normalizer48.

[0096] The predictor 47 calculates the pixel data (at the location ofinterest) of the HD signal to be produced, using data (SD pixel data) xiof the prediction taps selectively extracted by the first tap selector41 and the coefficient data Wi read from the coefficient memory 53, inaccordance with the prediction equation given by equation (4).

[0097] As described above, to convert a SD signal (525i signal) into aHD signal (1050i signal), it is required to obtain four pixels (forexample, pixels HD1 to HD4 shown in FIG. 3 or HD1′ to HD4′ shown in FIG.4) of the HD signal corresponding to each pixel of the SD signal. Thus,the predictor 47 generates pixels data of the HD signal on ablock-by-block basis wherein each block includes 2×2 pixels. Morespecifically, prediction tap data xi corresponding to the four pixels(of interest) in a unit pixel block are supplied from the first tapselector 41 to the predictor 47, and coefficient data Wi correspondingto the four pixels in that unit pixel block are supplied from thecoefficient memory 53 to the predictor 47, and the predictor 47calculates, separately, each of data y1 to y4 of the four pixels in theunit pixel block in accordance with the above-described predictionequation given by equation (4).

[0098] The normalizer 48 normalizes the data y1 to y4 of the four pixelssequentially output from the predictor 47 by dividing them by thenormalization coefficients Sn which are read from the normalizationcoefficient memory 55 and which correspond to the coefficient data Wi(i=1 to n) used in the calculations. As described above, the coefficientgenerator 52 determines coefficient data Wi used in the predictionequation. However, the determined coefficient data can includes roundingerrors, and thus it is not guaranteed that the sum of the coefficientdata Wi (i=1 to n) is equal to 1.0, and the data y1 to y4 of therespective pixels calculated by the predictor 47 have fluctuations inlevel due to the rounding errors. To avoid the above problem, thenormalizer 48 normalizes the data y1 to y4 thereby removing thefluctuations in level due to the rounding errors.

[0099] The data y1′ to y4′ of four pixels in each unit pixel blockproduced by the normalizer 48 by normalizing the data y1 to y4 aresequentially supplied to a post processing unit 49. The post processingunit 49 rearranges the received data into a line-sequential fashion andoutputs the resultant data in the 1050i format.

[0100] The feature value extractor 56 extracts image feature values suchas a variance or mean value of image levels from the SD signal (525isignal) stored in the buffer memory 14 and supplies the extracted imagefeature values to the history information memory 50 together withenvironmental information indicating temperature, humidity, and/orbrightness of a light in an ambient, detected by the sensor 20 andsupplied to the feature value extractor 56 via the system controller 12and also together with information such as a title, actor/actress'snames, and a category of the content from the information supplied fromthe system controller 12.

[0101] The values of parameters S and Z input from the system controller12 to the coefficient generator 52 are also input to the historyinformation memory 50. In response, the history information memory 50calculates the volume values Sv and Zv corresponding to the feature datasupplied from the feature value extractor 56. The calculated volumevalues Sv and Zv are output to the coefficient generator 54.

[0102]FIG. 5 is a block diagram showing the details of the historyinformation memory 50. The history information memory 50 includes afeature value quantizer 61, a weight calculator 62, a cumulative weightmemory 63, and a volume value generator 64.

[0103] The feature value quantizer 61 receives, from the feature valueextractor 56, image feature values such as the variance or mean value ofimage levels, environmental information indicating temperature,humidity, and/or brightness of the light in the ambient, and/orinformation such as the title, actor/actress's names, and the categoryof the content. In a case in which information in the form of a numeralis input to the feature value quantizer 61, the feature value quantizer61 quantizes the input information with predetermined quantizationsteps. However, in a case in which the input information is not in theform of a numeral, the feature value quantizer 61 quantizes the inputinformation by classifying it into one of predetermined groups. Thequantized feature value v is output to the cumulative weight memory 63and the volume value generator 64.

[0104] The feature value quantizer 61 may receive a value of one featureand quantize the received value of the feature, or may receive values ofa plurality of features and quantize the received values of theplurality of features.

[0105] If the user operates the remote commander 2 to adjust the imagequality, and if signals indicating the changed parameters S and Z areinput to the weight calculator 62 via the system controller 12, theweight calculator 62 calculates the weight d in accordance with thevolume control operation performed on the remote commander 2 by the useruntil the volume control operation is ended. The value of the weight dis repeatedly calculated until the volume control operation by the useris ended. If the volume control operation by the user is completed, theweight calculator 62 outputs the value of the weight d to the cumulativeweight memory 63 and the volume value generator 64.

[0106] A specific example of the manner of calculating the weight d isto assume that a user spends a longer time to perform the volume controloperation increases when the user wants to make a finer adjustment, andto assign, to the weight d, a value proportional to the time spent bythe user to perform the volume control operation. Another example of themanner of calculating the weight d is to assume that volume values canbe regarded as having been precisely set when the volume values haveconverged quickly in the volume control operation but volume values canbe regarded as not being set well when volume values have convergedslowly, and thus to assign the weight d a value given as a function ofthe mean value of the square of the absolute value of the volumeadjustment range and the square of the median. Another example of themanner of calculating the weight d is to assume that the user does notlike the image quality at a time immediately before the user starts thevolume control operation and to assign, to the weight d, a small valuewhen an adjusted value is close to a value of a parameter at that timeimmediately before the user starts the volume control operation.

[0107] The cumulative weight memory 63 stores cumulative weights Dv forrespective input feature values v, and extracts a cumulative weight Dvcorresponding to a quantized feature value v input from the featurevalue extractor 56 and outputs the extracted cumulative weight Dv to thevolume value generator 64. After outputting the cumulative weight Dv tothe volume value generator 64, the cumulative weight memory 63calculates a new cumulative weight Dv using the weight d input from theweight calculator 62 in accordance with an equation Dv=Dv+d and replacesthe current value of the cumulative weight Dv with the calculatedcumulative weight Dv.

[0108] In the volume value generator 64, the volume values S′v and Z′voutput in the past are stored. If the volume value generator 64 receivesthe parameters S and Z indicating the final adjusted values from thecontroller 12, the quantized feature value v from the feature valueextractor 56, the weight d from the weight calculator 62, and thecumulative weight Dv from the cumulative weight memory 63, the volumevalue generator 64 calculates the output volume values Sv and Zvcorresponding to the feature value v using the above received values inaccordance with equations (7) and (8). The calculated volume values Svand Zv are stored in the volume value generator 64. Under the control ofthe system controller 12, the volume value generator 64 supplies theoutput volume values Sv and Zv to the coefficient generator 52.

Sv=((S′v×Dv)+S)/(Dv+d)  (7)

Zv=((Z′v×Dv)+Z)/(Dv+d)  (8)

[0109] If readjustment of the parameters S and Z is started, the volumevalue generator 64 stores the output volume values Sv and Zv calculatedin accordance with equations (7) and (8) as the past output volumevalues S′v and Z′v, and the volume value generator 64 calculates newoutput volume values Sv and Zv.

[0110] The cumulative weight memory 63 and the volume value generator 64are formed of, for example, nonvolatile memory so that the contentsstored therein can be retained even when the power of the television set1 is in the off state.

[0111] Thus, as described above, on the basis of features of the image,environmental parameters, and the adjustment values used in the past,the history information memory 50 calculates volume values which arelikely to correspond to image quality desired by the user, and thehistory information memory 50 outputs the calculated volume values tothe coefficient generator 52 for use in generation of coefficients whichdetermine the image quality in response to the volume control operationby the user.

[0112] Referring again to FIG. 1, the construction of the television set1 is further described below.

[0113] An OSD (On Screen Display) processing unit 16 generates a displaysignal for displaying characters or graphical images on the screen of adisplay 18. A mixer 17 combines the display signal output from the OSDprocessing unit 16 with the HD signal output from the image signalprocessing unit 15 and supplies the resultant signal to the display 18.As for the display 18, for example, a CRT (cathode-ray tube) display ora LCD (liquid crystal display) may be used. The display 18 displays animage corresponding to the HD signal output from the image signalprocessing unit 15 and also displays, as required, an imagecorresponding to the display signal mixed by the mixer 17.

[0114] As required, a drive 19 is connected to the system controller 12and a magnetic disk 21, an optical disk 22, a magnetooptical disk 23, ora semiconductor memory 24 is mounted on the driver 19 to install acomputer program into the system controller 12.

[0115] The operation of the television set 1 is described below.

[0116] In response to a command input by a user via the remote commander2, the system controller 12 controls the tuner 13. Under the control ofthe system controller 12, the tuner 13 performs channel selection,intermediate frequency amplification, and detection on a broadcastsignal received via the antenna 3. The resultant SD signal (525i signal)is output to the buffer memory 14.

[0117] The SD signal (525i signal) supplied from the tuner 13 to thebuffer memory 14 is temporarily stored in the buffer memory 14. The SDsignal temporarily stored in the buffer memory 14 is then supplied tothe image signal processing unit 15 and converted to a HD signal (1050isignal) in accordance with a control signal supplied from the systemcontroller 12.

[0118] That is, the image signal processing unit 15 acquires pixel dataof a HD signal (hereinafter, referred to as “HD pixel data”) from pixeldata of the SD signal (hereinafter, referred to as “SD pixel data”). TheHD signal output from the image signal processing unit 15 is combined,as required, by the mixer 17 with a display signal indicating charactersor a graphical image output from the OSD processing unit 16. Theresultant signal is supplied to the display 18, and an image isdisplayed on the screen of the display 18.

[0119] The user can adjust the spatial and temporal resolutions of theimage displayed on the screen of the display 18 by operating the remotecommander 2. The image signal processing unit 15 calculates HD pixeldata in accordance with the prediction equation. In this calculation,coefficients in the prediction equation are determined on the basis ofthe parameters S and Z indicating the spatial and temporal resolutionsspecified by the user via the remote commander 2, or determined on thebasis of, instead of the parameters S and Z, the volume values Sv and Zvcalculated by the history information memory 50, and the HD pixel dataare calculated in accordance with the generation equation including theparameters S and Z. As a result, the image of the HD signal output fromthe image signal processing unit 15 has the spatial resolution andtemporal resolution corresponding to the specified parameters S and Z orcalculated volume values Sv and Zv, respectively.

[0120]FIG. 6 shows an example of a user interface for adjusting theparameters S and Z. During the adjustment, an adjustment screen 71 isdisplayed in the OSD manner on the display 18. On the adjustment screen71, as shown in FIG. 6, a star-shaped icon 72 is displayed to indicatethe adjusted values of the parameters S and Z. The remote commander 2includes a joystick 81 operated by the user and an automatic adjustmentbutton 82.

[0121] The user can move the location of the icon 72 on the adjustmentscreen 71 by operating the joystick 81 to specify the values of theparameters S and Z which determine the spatial and temporal resolution,respectively, such that the HD image has desired image quality. If theuser presses the automatic adjustment button 82, the coefficients in theprediction equation is calculated using the volume values Sv and Zvcalculated by the history information memory 50, and the image qualityof the HD pixel data is automatically adjusted in accordance with thepreference of the user, the features of the image, and the environmentalparameters so that the image displayed has desirable image quality.

[0122]FIG. 7 shows, in an enlarged fashion, a part of the adjustmentscreen 71 shown in FIG. 6. If the icon 72 is moved to right or left, theparameter Z associated with the temporal resolution is adjusted (inaccordance with a new value on the horizontal coordinate axis shown inFIG. 7). On the other hand, if the icon 72 is moved upward or downward,the parameter S associated with the spatial resolution is adjusted (inaccordance with a new value on the vertical coordinate value shown inFIG. 7). Thus, the user can easily adjust the parameters S and Z bymoving the icon 72 on the adjustment screen 71 displayed on the display18.

[0123] The remote commander 2 may include, instead of the joystick 81, amouse, a trackball, or another type of pointing device. Numeralsindicating the values of the parameters S and Z adjusted by the user maybe displayed on the adjustment screen 71.

[0124] The operation of the image signal processing unit 15 is describedbelow.

[0125] If the second tap selector 42 receives the SD signal (525isignal) stored in the buffer memory 14, the second tap selector 42selectively extracts spatial class tap data (SD pixel data) at locationsin the vicinity of four pixels (at locations of interest) in a unitpixel block of a HD signal (1050i signal) to be produced. The spatialclass tap data (SD pixel data) selectively extracted by the second tapselector 42 is supplied to the spatial class detector 44. The spatialclass detector 44 performs the ADRC processing on the respective SDpixel data given as the spatial class tap data thereby obtaining arequantized code qi serving as class information indicating a spatialclass (a class indicating a spatial waveform) (refer to equation (1)).

[0126] On the other hand, if the third tap selector 43 receives the SDsignal (525i signal) stored in the buffer memory 14, the third tapselector 43 selectively extracts motion class tap data (SD pixel data)at locations in the vicinity of four pixels (at locations of interest)in the unit pixel block of the HD signal (1050i signal) to be produced.The motion class tap data (SD pixel data) selectively extracted by thethird tap selector 43 is supplied to the motion class detector 45. Themotion class detector 45 obtains class information MV indicating amotion class (a class indicating the degree of motion) from therespective SD pixel data given as the motion class tap data.

[0127] The motion information MV and the requantized code qi aresupplied to the class combiner 46. The class combiner 46 obtains a classcode CL indicating a class of four pixels (of interest) in each unitpixel block of the HD signal (1050i signal) to be produced, from thesupplied motion information MV and the requantized code qi (refer toFIG. (3)). The resultant class code CL is supplied as read addressinformation to the coefficient memory 53 and the normalizationcoefficient memory 55.

[0128] The values of parameters S and Z specified by the user or thevolume values Sv and Zv calculated by the history information memory 50are input to the coefficient generator 52

[0129] A process of updating the volume values is described below withreference to a flow chart shown in FIG. 8.

[0130] In step S1, the cumulative weight memory 63 initializes the valueof the cumulative weight Dv corresponding to a feature value vindicating the cumulative weight, and the volume value generator 64initializes volume values Sv and Zv. The optimum number of quantizationsteps of feature values depends on the types of feature values and theclassification method. In the present embodiment, it is assumed that thefeature value quantizer 61 quantizes each feature value into one of Vsteps.

[0131] In step S2, the volume value generator 64 determines whether ornot to output volume values Sv and Zv on the basis of a control signalsupplied from the system controller 12. The volume values Sv and Zv areoutput from the volume value generator 64, for example, when a userissues a command, via the remote commander 2, to make an automaticadjustment using volume values corresponding to feature values or whenthe cumulative weight has become greater than a predetermined value.When volume values Sv and Zv are not output from the volume valuegenerator 64, the coefficient generator 52 generates coefficients usingadjustment values S and Z supplied from the system controller 12. In thecase in which volume values Sv and Zv are output from the volume valuegenerator 64, the coefficient generator 52 generates coefficients usingvolume values Sv and Zv output from the volume value generator 64.

[0132] If, in step S2, it is determined not to output volume values, theprocess proceeds to step S3. In step S3, the system controller 12determines, on the basis of a signal supplied from the signal receiver11, whether a user has started a volume control operation. If it isdetermined in step S3 that the volume control operation has not beenstarted, the process returns to step S2 to repeat step S2 and followingsteps.

[0133] However, it is determined in step S3 that the volume controloperation has been started, the process proceeds to step S4. In step S4,the system controller 12 outputs, to the history information memory 50,a control signal indicating that the volume control operation has beenstarted. The weight calculator 62 determines the weight d, for example,by assigning, to the weight d, a value proportional to the time spent bythe user to perform the volume control operation or by assigning, to theweight d, a value determined as a function of the mean value of thesquare of the absolute value of the volume adjustment range and thesquare of the median. The determined weight d is output to thecumulative weight memory 63. The volume value generator 64 stores thecurrent volume values Sv and Zv as S′v=Sv and Z′v=Zv. That is, thevolume value generator 64 stores past volume values corresponding tofeature values v. The time spent by the user to perform the volumecontrol operation is output from a timer disposed in the remotecommander and supplied to the weight calculator 62 via the systemcontroller 12. The timer for measuring the time spent by the user toperform the volume control operation may be disposed in the systemcontroller.

[0134] In step S5, the system controller 12 determines, on the basis ofa signal received from the signal receiver 11, whether the volumecontrol operation has been ended. If it is determined in step S5 thatthe volume control operation is not ended, the processing returns tostep S4 to perform step S4 and the following steps.

[0135] If it is determined in step S5 that the volume control operationhas been ended, the process proceeds to step S6. In step S6, the systemcontroller 12 outputs, to the history information memory 50, a controlsignal indicating that the volume control operation has been ended. Whenthe feature value quantizer 61 receives, from the feature valueextractor 56, image feature values indicating, for example, the varianceof image level, the mean value of image levels (associated withbrightness, chroma, and the like), environmental information indicatingtemperature, humidity, brightness of a light in the ambient, orinformation associated with the content such as a title, actor/actress'snames, and a category, the feature value quantizer 61 quantizes thereceived information with V steps and supplies the resultant quantizedfeature value v to the cumulative weight memory 63 and the volume valuegenerator 64.

[0136] In step S7, on the basis of a quantized feature value v suppliedfrom the feature value extractor 56, the volume value generator 64acquires a value indicating the cumulative weight Dv corresponding tothe feature value v from the cumulative weight memory 63.

[0137] In step S8, the volume value generator 64 acquires values ofparameters S and Z finally adjusted by a user. In step S9, on the basisof the parameter S and Z, the quantized feature value v supplied fromthe feature value extractor 56, the weight d supplied from the weightcalculator 62, and the cumulative weight Dv supplied from the cumulativeweight memory 63, the volume value generator 64 calculates output volumevalues Sv and Zv corresponding to the feature value v using equations(7) and (8) described above.

[0138] In step S10, the cumulative weight memory 63 calculates a newcumulative weight Dv using the weight d input from the weight calculator62 in accordance with an equation Dv=Dv+d and replaces the current valueof the cumulative weight Dv with the calculated cumulative weight Dv.After completion of step S10, the process returns to step S2 to repeatstep S2 and following steps.

[0139] If it is determined in step S2 that the volume values should beoutput, the process proceeds to step S11. In step S11, the feature valuequantizer 61 receives, from the feature value extractor 56, imagefeature values such as the variance or mean value of image levels,environmental information indicating temperature, humidity, brightnessof a light in the ambient, and/or information such as the title,actor/actress's names, and the category of the content, and the featurevalue quantizer 61 quantizes the received information with V steps. Thefeature value quantizer 61 supplies the resultant quantized featurevalue v to the volume value generator 64.

[0140] In step S12, the volume value generator 64 extracts volume valuesSv and Zv corresponding to the received feature value v from the volumevalues stored for respective feature values, and the volume valuegenerator 64 outputs the extracted volume values Sv and Zv to thecoefficient generator 52. After the completion of step S12, theprocessing proceeds to S3 to perform step S3 and following steps.

[0141] As described above with reference to FIG. 8, the historyinformation memory 50 calculates the volume values corresponding to theinput image feature value in accordance with the weights and stores thecalculated volume values. As required, the history information memory 50outputs the volume values corresponding to the feature value to thecoefficient generator 52.

[0142] By producing volume values Sv and Zv in accordance with the imagefeature, the environmental parameters, and/or the information associatedwith the content data, it becomes possible to process the content datausing the volume values Sv and Zv corresponding to the image feature,the environmental parameters, and/or the information associated with thecontent data.

[0143] During each vertical blanking period, the coefficient generator52 determines coefficient data Wi (i=1 to n) of the prediction equationusing coefficient seed data w10 to wn9 for each combination of a classand output pixels (HD1 to HD4 or HD1′ to HD4′) corresponding to valuesof parameters S and Z adjusted by the user or volume values Sv and Zvreceived from the history information memory 50, and the coefficientgenerator 52 outputs the resultant coefficient data Wi to thecoefficient memory 53 to store them therein (refer to equation (5)).Furthermore, the normalization coefficient calculator 54 producesnormalization coefficients Sn corresponding to the coefficient data Wi(i=1 to n) of the prediction equation determined by the coefficientgenerator 52, and the resultant normalization coefficients Sn are storedin the normalization coefficient memory 55 (refer to equation (6)).

[0144] When the class code CL is supplied as read address information tothe coefficient memory 53, the coefficient memory 53 outputs coefficientdata Wi used in the prediction equation for four output pixels (HD1 toHD4 in an odd field or HD1′ to HD4′ in an even field) corresponding tothe class code CL. The output coefficient data Wi are supplied to thepredictor 47. When the first tap selector 41 receives the SD signal(525i signal) stored in the buffer memory 14, the first tap selector 41selectively extracts prediction tap data (SD pixel data) at locations inthe vicinity of four pixels (at locations of interest) in a unit pixelblock of a HD signal (1050i signal) to be produced.

[0145] On the basis of the prediction tap data (SD pixel data) xi andthe coefficient data Wi read from the coefficient memory 53 for fouroutput pixels, the predictor 47 calculates the data y1 to y4 of the fourpixels (at locations of interest) in the unit pixel block of the HDsignal to be produced (refer to equation (4)). The calculated data y1 toy4 of the four pixels in the unit pixel block of the HD signal aresequentially output from the predictor 47 to the normalizer 48.

[0146] As described above, when the class code CL is supplied as readaddress information to the normalization coefficient memory 55, thenormalization coefficient memory 55 outputs normalization coefficientsSn corresponding to the class code CL. That is, the normalizationcoefficients Sn, corresponding to the coefficient data Wi used in thecalculation of the HD pixel data y1 to y4 output from the predictor 47,are read from the normalization coefficient memory 55 and supplied tothe normalizer 48. The normalizer 48 normalizes the HD pixel data y1 toy4 output from the predictor 47 by dividing them by the correspondingnormalization coefficients Sn thereby removing the fluctuations inlevels of the data y1 to y4 due to the rounding errors in thecalculation of the coefficient data Wi performed by the coefficientgenerator 52. Thus, data y1′ to y4′ are obtained.

[0147] The resultant normalized data y1′ to y4′ produced in theabove-described manner for the four pixels in the unit pixel block aresequentially output from the normalizer 48 and supplied to the postprocessing unit 49. If the post processing unit 49 receives the data y1′to y4′ of the four pixels in each unit pixel block sequentially suppliedfrom the normalizer 48, the post processing unit 49 rearranges thereceived data into a line-sequential fashion and outputs the resultantdata in the 1050i format. Thus, the HD signal in the 1050i format isoutput from the post processing unit 49.

[0148] As described above, the image signal processing unit 15calculates the HD pixel data y using the adjusted values of theparameters S and Z or coefficient data Wi (i=1 to n) of the predictionequation corresponding to the volume values Sv and Zv calculated by thehistory information memory 50. Thus, as described above, the user canadjust the spatial and temporal resolutions of the image of the HDsignal by adjusting the values of the parameters S and Z. The user canalso make an automatic adjustment of the image quality by using thevolume values Sv and Zv stored in the history information memory 50 foreach feature value.

[0149] The coefficient data of each class corresponding to the adjustedvalues of parameters S and Z or the volume values Sv and Zv suppliedfrom the history information memory 50 are generated by the coefficientgenerator 52 whenever they are required, and thus a memory for storing alarge amount of coefficient data is not necessary. This allows areduction in the memory capacity.

[0150] Furthermore, as described above, the user can adjust the valuesof the parameters S and Z on the adjustment screen 71. Volume values Svand Zv corresponding to feature values v are calculated using the valuesof the respective parameters S and Z supplied to the coefficientgenerator 52 from the system controller 12 and are stored in the volumevalue generator 64 (FIG. 5) of the history information memory 50.

[0151] Furthermore, as described in further detail below, the historyinformation memory 50 may calculate adjustment values associated withthe image quality most preferred by the user on the basis of the volumevalues Sv and Zv stored in the volume value generator 64, and thehistory information memory 50 may output the calculated adjustmentvalues to the coefficient generator 52 thereby making it possible toautomatically adjust the image quality.

[0152] That is, as shown in FIG. 9, the weight calculator 62 determinesthe mean values of the volume values Sv and Zv from the distributionthereof stored in the volume value generator 64. The weight calculator62 assumes that the calculated mean values indicate the adjustmentvalues most preferred by the user, and the weight calculator 62 outputsthe calculated mean values to the volume value generator 64. The volumevalue generator 64 transfers the received mean values of the volumevalues to the coefficient generator 52 to adjust the image quality onthe basis of the mean values.

[0153] Alternatively, instead of the simple mean value described above,a weighted mean value may be employed. More specifically, as shown inFIG. 10, the weight calculator 62 may weight the volume values Sv and Zvstored in the volume value generator 64 such that medians thereof aremost greatly weighted and may calculate the weighted mean value. Theweight calculator 62 assumes that the resultant weighted mean valuesindicate the adjustment values most preferred by the user, and theweight calculator 62 outputs the weighted mean values to the volumevalue generator 64. The volume value generator 64 transfers the receivedweighted mean values to the coefficient generator 52 to adjust the imagequality on the basis of the mean values.

[0154] If the weighted mean values are used, when the userunintentionally performs a volume control operation in an undesirablemanner, influences of such an operation on the adjustment are minimizedand the user can easily make the adjustment in a really desirablemanner.

[0155] When the image signal processing unit 15 or the circuit boardincluding the image signal processing unit 15 is replaced to upgrade thetelevision set 1, the volume values Sv and Zv stored as historyinformation in the volume value generator 64 of the history informationmemory 50 can be used to produce coefficient seed data w10 to wn9 to bestored in the information memory bank 51.

[0156] An example of the manner of generating coefficient seed data w10to wn9 is described below. In this example, coefficient seed data w10 town9 used as coefficient data in the generation equation (5) aredetermined.

[0157] For convenience in the following discussion, variables ti (i=0 to9) given by equation (9) are introduced.

t ₀=1, t ₁ =s, t ₂ =z, t ₃ =s ² , t ₄ =sz, t ₅ =z ² , t ₆ =s ³ , t ₇ =s² z, t ₈ =sz ² , t ₉ =z ³  (9)

[0158] By using equation (9), equation (5) can be rewritten as equation(10). $\begin{matrix}{W_{i} = {\sum\limits_{j = 0}^{9}{W_{ij}t_{j}}}} & (10)\end{matrix}$

[0159] Finally, undetermined coefficients wij are determined by means oflearning. That is, values of coefficients are determined from aplurality of SD pixel data and HD pixel data for each combination of aclass and output pixels so that the square error is minimized. Thistechnique is knows as the method of least square. Herein, let m denotethe number of data used in learning, ek denote the residual for the kthlearning data (1≦k≦m), and E denote the sum of square errors. Then E canbe expressed by equation (11) using equations (4) and (5). In equation(11), xik denotes kth pixel data at ith prediction tap location of a SDimage, and yk denotes kth pixel data, corresponding to xik, of a HDimage. $\begin{matrix}\begin{matrix}{E = {\sum\limits_{k = 1}^{m}e_{k}^{2}}} \\{= {\sum\limits_{k = 1}^{m}\left\lbrack {y_{k} - \left( {{W_{1}x_{1k}} + {W_{2}x_{2k}} + \cdots + {W_{n}x_{nk}}} \right)} \right\rbrack^{2}}} \\{= {\sum\limits_{k = 1}^{m}\left\{ {y_{k} - \left\lbrack {{\left( {{t_{0}w_{10}} + {t_{1}w_{11}} + \cdots + {t_{9}w_{19}}} \right)x_{1k}} + \cdots +} \right.} \right.}} \\\left. \left. {\left( {{t_{0}w_{n0}} + {t_{1}w_{n1}} + \cdots + {t_{9}w_{n9}}} \right)x_{nk}} \right\rbrack \right\}^{2} \\{= {\sum\limits_{k = 1}^{m}\left\{ {y_{k} - \left\lbrack {{\left( {w_{10} + {w_{11}s} + \cdots + {w_{19}z^{3}}} \right)x_{1k}} + \cdots +} \right.} \right.}} \\\left. \left. {\left( {w_{n0} + {w_{n1}s} + \cdots + {w_{19}z^{3}}} \right)x_{nk}} \right\rbrack \right\}^{2}\end{matrix} & (11)\end{matrix}$

[0160] In the method of least squares, wij are determined so thatpartial differentials of equation (11) with respect to wij become 0.That is, wij can be determined in accordance with equation (12).$\begin{matrix}{\frac{\partial E}{\partial W_{ij}} = {{\sum\limits_{k = 1}^{m}{2\left( \frac{\partial e_{k}}{\partial W_{ij}} \right)e_{k}}} = {{- {\sum\limits_{k = 1}^{m}{2t_{j}x_{ik}e_{k}}}} = 0}}} & (12)\end{matrix}$

[0161] If Xipjq and Yip are defined by equations (13) and (14), equation(12) can be rewritten as equation (15) using a matrix. $\begin{matrix}{X_{{ip}\quad {jq}} = {\sum\limits_{k = 1}^{m}{x_{ik}t_{p}x_{jk}t_{q}}}} & (13) \\{Y_{ip} = {\sum\limits_{k = 1}^{m}{x_{ik}t_{p}y_{k}}}} & (14) \\{{\begin{bmatrix}x_{1010} & x_{1011} & x_{1012} & \cdots & x_{1019} & x_{1020} & \cdots & x_{10{n9}} \\x_{1110} & x_{1111} & x_{1112} & \cdots & x_{1119} & x_{1120} & \cdots & x_{11{n9}} \\x_{1210} & x_{1211} & x_{1212} & \cdots & x_{1219} & x_{1220} & \cdots & x_{12{n9}} \\\vdots & \vdots & \vdots & ⋰ & \vdots & \vdots & ⋰ & \vdots \\x_{1910} & x_{1911} & x_{1912} & \cdots & x_{1919} & x_{1920} & \cdots & x_{19{n9}} \\x_{2010} & x_{2011} & x_{2012} & \cdots & x_{2019} & x_{2020} & \cdots & x_{20{n9}} \\\vdots & \vdots & \vdots & ⋰ & \vdots & \vdots & ⋰ & \vdots \\x_{n910} & x_{n911} & x_{n912} & \cdots & x_{n919} & x_{n920} & \cdots & x_{n9n9}\end{bmatrix}\begin{bmatrix}w_{10} \\w_{11} \\w_{12} \\\vdots \\w_{19} \\w_{20} \\\vdots \\w_{n9}\end{bmatrix}} = \begin{bmatrix}Y_{10} \\Y_{11} \\Y_{12} \\\vdots \\Y_{19} \\Y_{20} \\\vdots \\Y_{n9}\end{bmatrix}} & (15)\end{matrix}$

[0162] This equation is generally called a normal equation. This normalequation can be solved with respect to wij using, for example, thesweeping-out method (Gauss-Jordan's elimination method) therebyobtaining the coefficient seed data.

[0163]FIG. 11 shows the concepts of the method of generating thecoefficient seed data.

[0164] A plurality of SD signals are generated from a HD signal. Forexample, a total of 81 SD signals are generated by varying, in 9 steps,the parameters S and Z determining the spatial band (in the vertical andhorizontal directions) and the temporal band (in the frame-to-framedirection) of the filter used in generating the SD signals from the HDsignal, and learning is performed between the generated SD signals andthe HD signal thereby generating coefficient seed data.

[0165]FIG. 12 is a block diagram showing the construction of acoefficient seed data generator 121 for generating coefficient seed dataw10 to wn9 to be stored in the information memory bank 51 of thetelevision set 1.

[0166] A HD signal (1050i signal) used as a teacher signal is input viaan input terminal 141. A SD signal generator 143 thins out the HD signalin the vertical and horizontal directions in accordance with the historyinformation and the parameters S and Z input via the input terminals142, thereby obtaining a SD signal used as a student signal.

[0167] In accordance with the parameters S and Z input via the inputterminals 142, the SD signal generator 143 varies the spatial andtemporal bands of a band-limiting filter used in conversion from the HDsignal to the SD signal.

[0168] In the above process, the volume values Sv and Zv stored in thevolume value generator 64 of the history information memory 50 are inputas the history information to the SD signal generator 143.

[0169] When coefficient seed data w10 to wn9, to be stored in theinformation memory bank 51 of a television set 1 going to be used forthe first time, are generated, no history information is input to the SDsignal generator 143 because no history information is yet stored in thevolume value generator 64 of the history information memory 50.

[0170] That is, history information is input to the SD signal generator143, for example, to generate coefficient seed data w10 to wn9 to bestored in the information memory bank 51 when the image signalprocessing unit 15 or the circuit board including the image signalprocessing unit 15 is replaced to upgrade the television set 1.

[0171] In the image signal processing unit 15 described earlier withreference to FIG. 1, the terminals 142 are connected to the systemcontroller 12 so that the coefficient seed data generator 121 and thehistory information memory 50 can communicate with each other. Morespecifically, the terminal 161 of the input terminals 142 is connectedto the history information memory 50 so that the coefficient seed datagenerator 121 can receive history information.

[0172] The SD signal generator 143 adjusts the values of the inputparameters S and Z on the basis of the history information and variesthe spatial and temporal bands depending on the adjusted parameters Sand Z. In the case in which no history information is input, the spatialand temporal bands are varied directly depending on the input values ofthe parameters S and Z.

[0173] In the television set 1, the values of the parameters S and Z areadjusted within the range from 0 to 8 in predetermined steps in responseto a control operation performed by a user to adjust the spatial andtemporal resolutions.

[0174] In this case, if the spatial and temporal bands are varied in theSD signal generator 143 directly depending on the input values of theparameters S and Z, then in the television set 1 coefficient seed dataw10 to wn9 are generated so that the resolutions can be adjusted withinthe range represented by a solid line BF in FIG. 13 (within the rangefrom y1 to y2 for the spatial resolution and within the range from x1 tox2 for the temporal resolution).

[0175] In the case in which history information is input, the SD signalgenerator 143 determines the location of the barycenter of the frequencydistribution of parameters S and Z. In this case, for a particularnumber of newest values of parameters S and Z, newer values are weightedby greater factors. On the basis of the location of the barycenter, theSD signal generator 143 adjusts the input values of parameters S and Z.In this case, the bandwidth are reduced with increasing values ofparameters S and Z. Thus, in the television set 1 having the adjustedparameters, the resolutions are increased with increasing values ofparameters S and Z.

[0176] The input values of parameters S and Z are converted by means oflinear transformation such that the center of the allowable range ofparameters S and Z adjusted in the television set 1 is moved to thelocation of the barycenter determined above. For example, when thecenter values of the ranges of parameters S and Z adjusted in thetelevision set 1 are S0 and Z0, respectively, and the locations ofbarycenter are Sm and Zm, respectively, if the input values ofparameters S and Z are S1 and Z1, then the adjusted values S2 and Z2 ofparameters S and Z are given by the following conversion equations (16)and (17).

S ₂ =S ₁+(S _(m) −S ₀)  (16)

Z ₂ =Z ₁+(Z _(m) −Z ₀)  (17)

[0177] If the spatial and temporal bands are varied depending on thevalues of parameters S and Z adjusted in the above-descried manner,then, in the television set 1, coefficient seed data w10 to wn9 aregenerated so as to allow the adjustment of resolutions to be performedwithin a range denoted by a single-dotted line AF shown in FIG. 13(within a range from y1′ to y2′ for the spatial resolution and from x1′to x2′ for the temporal resolution) centered at the barycenter oflocations (denoted by symbols “x”) at which resolutions have beenadjusted within a range denoted by a solid line BF in FIG. 13.

[0178] In the process described above, when the location of barycenterof the frequency distribution of each of parameters S and Z isdetermined, the predetermined number of new values of parameters S and Zare weighted such that greater weighting factors are assigned to newervalues. However, it is not necessarily needed to weight the values. Thatis, the location of barycenter may be determined for unweighted values.Still alternatively, the location of barycenter may be simply determinedfor weighted values of a predetermined number of new values of each ofparameters S and Z without taking account the frequency distribution.Still alternatively, values of parameters S and Z having the greatestcounts in the frequency distribution of the values of parameters S and Zmay be determined, and those values may be used instead of the locationof barycenter. Still alternatively, the newest values of parameters Sand Z of predetermined numbers of new values of parameters S and Z maybe used instead of the location of barycenter.

[0179] Referring again to FIG. 12, the construction of the coefficientseed data generator 121 is further described below.

[0180] A first tap selector 144, a second tap selector 145, and a thirdtap selector 146 selectively extract data of a plurality of SD pixels inthe vicinity of a location of interest in the HD signal (1050i signal)from the SD signal (525i signal) output from the SD signal generator143, and the first to third tap selectors output the extracted SD pixeldata. Those first to third tap selectors 144, 145, and 146 areconstructed in a similar manner to the first to third tap selectors 41,42, and 43 of the image signal processing unit 15 described earlier withreference to FIG. 1.

[0181] The spatial class detector 147 detects the level distributionpattern of the spatial class tap data (SD pixel data) selectivelyextracted by the second tap selector 145. The spatial class detector 147determines a spatial class on the basis of the detected leveldistribution pattern and outputs class information indicating thedetermined motion class. The spatial class detector 147 is constructedin a similar manner to the spatial class detector 44 of the image signalprocessing unit 15 described earlier with reference to FIG. 1. Thespatial class detector 147 outputs, as class information indicating aspatial class, a requantized code qi of each SD pixel data employed asspatial class tap data.

[0182] The motion class detector 148 detects a motion class, chieflyindicating the degree of motion, from the data of motion class taps (SDpixel data) selectively extracted by the third tap selector 146, and themotion class detector 148 outputs the resultant class information MV.The motion class detector 148 is constructed in a similar manner to themotion class detector 45 of the image signal processing unit 15described earlier with reference to FIG. 1. The motion class detector148 calculates interframe differences from the motion class tap data (SDpixel data) selectively extracted by the third tap selector 146. Themotion class detector 148 compares the mean value of the absolute valuesof differences with threshold values thereby detecting a motion class,which is a measure of the degree of motion.

[0183] On the basis of the requantized code qi output from the spatialclass detector 147 as the class information indicating the spatial classand class information MV indicating the motion class output from themotion class detector 148, the class combiner 149 obtains a class codeCL indicating a class of the pixel of interest of the HD signal (1050isignal). The class combiner 149 is constructed in a similar manner tothe class combiner 46 of the image signal processing unit 15 describedearlier with reference to FIG. 1.

[0184] A normal equation generator 150 generates a normal equation(equation (15)) used to obtain coefficient seed data w10 to wn9, foreach class, from each HD pixel data y at a location of interest in theHD signal supplied via the input terminal 141, prediction tap data (SDpixel data) xi corresponding to the HD pixel data y and selectivelyextracted by the first tap selector 144, values of parameters S and Z,and a class code CL corresponding to the HD pixel data y and output fromthe class combiner 149.

[0185] In this case, learning data including a combination of one HDpixel data y and corresponding n prediction tap data (SD pixel data) xiis produced such that a plurality of SD signals are sequentiallyproduced by the SD signal generator 143 while varying the spatial bandand the temporal band depending on the variations in adjusted values ofparameters S and Z thereby producing learning data including a HD signaland the plurality of SD signals. The normal equation generator 150produces normal equations associated with a large number of learningdata corresponding to various values of parameters S and Z, wherebycoefficient seed data w10 to wn9 can be determined from the normalequations.

[0186] In the above process, after learning data including a combinationof one HD pixel data y and corresponding n prediction tap data (SD pixeldata) xi is produced, the normal equation generator 150 produces anormal equation for each of output pixels (HD1 to HD4 shown in FIG. 3 orHD1′ to HD4′ shown in FIG. 4). For example, a normal equationcorresponding to HD1 is produced from learning data including HD pixeldata y at a similar location relative to the central prediction tap tothat of HD1.

[0187] The data of the normal equation generated by the normal equationgenerator 150 for each combination of a class and output pixels issupplied to a coefficient seed data determination unit 151. Thecoefficient seed data determination unit 151 solves the normal equationusing, for example, the sweeping-out method, thereby determiningcoefficient seed data w10 to wn9 for each combination of a class andoutput pixels. The coefficient seed data determined by the coefficientseed data determination unit 151 is stored in a coefficient seed datamemory 152. As required, an input/output interface 153 is connected toanother device (such as the information memory bank 51 of the imagesignal processing unit 15 described earlier with reference o FIG. 1) andthe coefficient seed data stored in the coefficient seed data memory 152is output.

[0188] Now, the operation of the coefficient seed data generator 121shown in FIG. 12 is described below.

[0189] A HD signal (1050i signal) used as a teacher signal is input viaan input terminal 141. A SD signal generator 143 thins out the HD signalin the vertical and horizontal directions thereby obtaining a SD signal(525i signal) used as a student signal.

[0190] In the above process, values of parameters which determine thespatial band and the temporal band of the band-limiting filter used inthe generation of the SD signals from the HD signal, that is, parametersS and Z which determine the spatial resolution and the temporalresolution of the SD signals to be produced, are input to the SD signalgenerator 143.

[0191] Furthermore, when the image signal processing unit 15 or thecircuit board including the image signal processing unit 15 is replacedto upgrade the television set 1, parameters S and Z input in the past bythe user and stored as history information in the volume value generator64 of the history information memory 50 of the original image signalprocessing unit 15 may be input via the input terminal 142 to the SDsignal generator 143 to produce coefficient seed data w10 to wn9 to bestored in the information memory bank 51.

[0192] If history information is input, the SD signal generator 143adjusts the values of the input parameters S and Z on the basis of thehistory information. For example, the location of the barycenter ofparameters S and Z is determined from the input history information, andthe input values of parameters S and Z are converted by means of alinear transformation such that the center of the allowable range ofparameters S and Z adjusted in the television set 1 is moved to thelocation of the barycenter determined above. In accordance with theadjusted values of parameters S and Z, the SD signal generator 143varies the spatial and temporal bands of a band-limiting filter used inconversion from the HD signal to the SD signal.

[0193] When coefficient seed data w10 to wn9, to be stored in theinformation memory bank 51 of a television set 1 going to be used forthe first time, are generated, no history information is input. In thiscase, the spatial and temporal bands of the band-limiting filter used ingeneration of the SD signal from the HD signal are varied directlydepending on the input values of the parameters S and Z.

[0194] By changing stepwise the values of parameters S and Z input tothe SD signal generator 143 in predetermined steps, the spatial andtemporal bands of the band-limiting filter used in generating the SDsignal from the HD signal are varied, thereby generating a plurality ofSD signals whose spatial and temporal bands vary stepwise.

[0195] Furthermore, from the SD signal (525i signal) generated by the SDsignal generator 143, the second tap selector 145 selectively extractsspatial class tap data (SD pixel data) at locations in the vicinity ofthe location of interest in the HD signal (1050i signal). The spatialclass tap data (SD pixel data) selectively extracted by the second tapselector 145 is supplied to the spatial class detector 147. The spatialclass detector 147 performs the ADRC processing on the respective SDpixel data given as the spatial class tap data thereby obtaining arequantized code qi serving as class information indicating a spatialclass (a class indicating a spatial waveform) (refer to equation (1)).

[0196] From the SD signal generated by the SD signal generator 143, thethird tap selector 146 selectively extracts motion tap data (SD pixeldata) at locations in the vicinity of the location of interest in the HDsignal. The motion class tap data (SD pixel data) selectively extractedby the third tap selector 146 is supplied to the motion class detector148. The motion class detector 148 obtains class information MVindicating a motion class (a class indicating the degree of motion) fromthe respective SD pixel data given as the motion class tap data.

[0197] The class information MV and the requantized code qi are suppliedto the class combiner 149. On the basis of the supplied classinformation MV and requantized code qi, the class combiner 149determines a class code CL indicating a class of pixel data at thelocation of interest in the HD signal (1050i signal) (refer to equation(3)).

[0198] From the SD signal generated by the SD signal generator 143, thefirst tap selector 144 selectively extracts prediction tap data (SDpixel data) at locations in the vicinity of the location of interest inthe HD signal.

[0199] A normal equation generator 150 generates a normal equation(equation (15)) used to obtain coefficient seed data w10 to wn9, foreach class, from each HD pixel data y at a location of interest in theHD signal supplied via the input terminal 141, prediction tap data (SDpixel data) xi corresponding to the HD pixel data y and selectivelyextracted by the first tap selector 144, values of parameters S and Z,and a class code CL corresponding to the HD pixel data y and output fromthe class combiner 149.

[0200] The coefficient seed data determination unit 151 solves eachnormal equation thereby determining coefficient seed data w10 to wn9 foreach combination of a class and output pixels. The obtained coefficientseed data w10 to wn9 are stored in the coefficient seed data memory 152.As required, the coefficient seed data memory 152 are output to theoutside via the input/output interface 153.

[0201] As described above, the coefficient seed data generator 121 shownin FIG. 12 can generate coefficient seed data w10 to wn9 to be stored inthe information memory bank 51 of the image signal processing unit 15shown in FIG. 1. The generated coefficient seed data are used ascoefficients, for each combination of a class and output pixels (HD1 toHD4 or HD1′ to HD4′), in the generation equation (5) to determinecoefficient data Wi used in the prediction equation.

[0202] When the image signal processing unit 15 or the circuit boardincluding the image signal processing unit 15 is replaced to upgrade thetelevision set 1, volume values Sv and Zv stored in the volume valuegenerator 64 of the history information memory 50 of the television set1 are input via the input terminal 142 to the SD signal generator 143 toproduce coefficient seed data w10 to wn9 to be stored in the informationmemory bank 51.

[0203] The SD signal generator 143 adjusts the input values of theparameters S and Z on the basis of the history information and variesthe spatial and temporal bands of the band-limiting filter used ingenerating the SD signal from the HD signal using the adjusted values ofthe parameters S and Z.

[0204] If the coefficient seed data w10 to wn9 obtained in theabove-described manner are stored in the information memory bank 51 inan image signal processing unit 15 or a circuit board including theimage signal processing unit 15 newly installed in the television set 1when the television set 1 is upgraded, it becomes possible for a user toadjust the resolutions within a range (represented by a single-dottedline AF in FIG. 13) centered at the location of barycenter ofresolutions used in the past, by adjusting the values of the parametersS and Z. That is, the allowable resolution range is automatically set inaccordance with the preference of the user, and the user can adjust theresolutions within the range.

[0205] Another embodiment is described below with reference to FIG. 14.

[0206]FIG. 14 is a block diagram showing the construction of atelevision set 171. In FIG. 14, similar parts to those in FIG. 1 aredenoted by similar reference numerals, and they are not described infurther detail herein.

[0207] The television set 171 is similar to the television set 1described earlier with reference to FIG. 1 except that the image signalprocessing unit 15 shown in FIG. 1 is replaced with an image signalprocessing unit 175, and the OSD processing unit 16 is replaced with anOSD processing unit 182. The image signal processing unit 175 is similarto the image signal processing unit 15 described earlier with referenceto FIG. 1 except that the history information memory 50 is replaced witha history information memory 181 and that the feature value extractor 56is removed. In this television set 171, as in the television set 1,coefficient seed data w10 to wn9 to be stored in the information memorybank 51 of the image signal processing unit 175 are generated by thecoefficient seed data generator 121 described above with reference toFIG. 12.

[0208] From the system controller 12, a control signal and parameters Sand Z, which are adjustment values (associated with, for example, aspatial resolution, a temporal resolution, and noise) are input to thehistory information memory 181 of the image signal processing unit 175.The history information memory 181 stores the received volume data. Thehistory information memory 181 automatically creates a new volume axison the basis of the stored volume data and outputs, as required,information associated with the new volume axis to the OSD processingunit 182. On the basis of the information associated with the new volumeaxis received from the history information memory 181, the OSDprocessing unit 182 generates OSD data associated with the new volumeaxis corresponding to the adjustment screen 71 described earlier withreference to FIGS. 6 and 7 and displays it on the display 18.

[0209]FIG. 15 is a block diagram showing the construction of the historyinformation memory 181.

[0210] If a volume value converter 191 receives input parameters S and Zindicating preference of a user from the system controller 12, thevolume value converter 191 converts the parameters S and Z into volumevalues S′ and Z′ in accordance with a conversion table stored in aconversion table memory 194. The resultant volume values S′ and Z′ aresupplied to a coefficient generator 52 or a volume value storing unit192. If the volume axis is changed as will be described later, thevolume value converter 191 outputs new axis information used to producedata indicating the locations, on the new volume axes on the adjustmentscreen, of the volume values S′ and Z′ corresponding to the adjustmentvalues input by the user and supplied to the volume value converter 191via the system controller 12 and also outputs volume values S′ and Z′corresponding to the new volume axes. The output data are supplied tothe OSD processing unit 182.

[0211] If the volume value storing unit 192 receives the volume valuesS′ and Z′ from the volume value converter 191, the volume value storingunit 192 stores a predetermined number of sets (for example, 20 sets) ofvolume values S′ and Z′. The stored data is used to produce theconversion table. The stored data is also used as history information ingenerating coefficient seed data descried above.

[0212] As required, a conversion table calculator 193 calculates aconversion table corresponding to a volume axis on the basis of the dataS′ and Z′ stored in the volume value storing unit 192 and outputs theresultant conversion table to the conversion table memory 194. Theconversion table memory 194 stores the received conversion table. Theconversion table may be updated, for example, when a user issues acommand via the remote commander 2 to use a new volume axis or when agreater number of sets of volume values than a predetermined value arestored in the volume value storing unit 192. If the conversion tablememory 194 receives the updated conversion table from the conversiontable calculator 193, the conversion table memory 194 stores thereceived conversion table.

[0213] If the OSD processing unit 182 shown in FIG. 14 receives the newaxis information and the converted volume values S′ and Z′ from thevolume value converter 191, the OSD processing unit 182 generates OSDdata for displaying an icon 72 at a location corresponding to the volumevalues being currently adjusted by the user, on an adjustment screensimilar to the adjustment screen 71 described earlier with reference toFIGS. 6 and 7. The generated OSD data is output to the mixer 17 anddisplayed on the display 18.

[0214] The process of updating the volume axis is described below withreference to a flow chart shown in FIG. 16.

[0215] In step S21, the volume value storing unit 192 initializes thevolume values stored therein, and the conversion table memory 194initializes the conversion table stored therein.

[0216] In step S22, the conversion table calculator 193 determineswhether to update the conversion table, for example, depending onwhether the conversion table calculator 193 receives a control signalindicating that a user has issued via the remote commander 2 a commandto use a new volume axis or whether it is detected that a greater numberof sets of volume values than the predetermined number are stored in thevolume value storing unit 192.

[0217] If it is determined in step S22 that the conversion table shouldbe updated (because the command has been issued by the user or a greaternumber of sets of volume values than the predetermined number are storedin the volume value storing unit 192), the process proceeds to step S23.In step S23, the conversion table calculator 193 calculates theconversion table using the data S′ and Z′ stored in the volume valuestoring unit 192 and outputs the resultant conversion table to theconversion table memory 194. The conversion table memory 194 replacesthe current conversion table with the received conversion table.

[0218] In a case in which volume adjustment values specified by a userhave a simple distribution in the initial volume axis space, forexample, such as that shown in FIG. 17 (in the example shown in FIG. 17,the distribution of volume adjustment values can be approximated by alinear line), the conversion table calculator 193 approximates thedistribution of the volume values by a linear expression using theprincipal component analysis or the like. In accordance with theapproximate linear expression, the conversion table calculator 193produces new volume axes as shown in FIG. 18 and calculates theconversion table representing the conversion between the initial volumeaxis space and the new volume axis space. The calculated conversiontable is output to the conversion table memory 194 and stored therein.

[0219] In the examples shown in FIGS. 17 and 18, a fitting line L isdetermined in the coordinate space in which the S and Z axes are used asvolume axes, and S′ axis and Z′ axis are determined as new volume axessuch that the fitting line L becomes parallel to the Z′ axis.

[0220] In the determination of new volume axes, instead of the principalcomponent analysis using linear approximation, approximation may beperformed by another method such as approximation using a higher-orderexpression or approximation using a VQ (Vector Quantization) table andVQ codes.

[0221] In the case in which it is determined in step S22 that it is notneeded to update the conversion table, the process proceeds to step S24.The process also proceeds to step S24 when step S23 is completed. Instep S24, the system controller 12 determines whether the user hasstarted a volume control operation, on the basis of a signal receivedvia the signal receiver 11.

[0222] If it is determined in step S24 that the volume control operationby the user is not started, the process returns to step S22 to repeatstep S22 and following steps.

[0223] However, if it is determined in step S24 that the volume controloperation by the user has been started, the process proceeds to stepS25. In step S25, the system controller 12 outputs a control signal tothe history information memory 181 to notify that the volume controloperation has been started, and the system controller 12 transfersparameters S and Z, which are parameters used to make an imageadjustment, to the volume value converter 191.

[0224] In step S26, the volume value converter 191 calculates the volumevalues S′ and Z′ using the parameters S and Z received from the systemcontroller 12 in accordance with the conversion table stored in theconversion table memory 194. Herein, if the conversion table stored inthe conversion table memory 194 is that initialized in step S21, thevolume values S′ and Z′ become equal to the parameters S and Z,respectively. The volume value converter 191 outputs the convertedvolume values S′ and Z′ to the coefficient generator 52. Furthermore,the volume value converter 191 outputs the volume values S′ and Z′ tothe OSD processing unit 182. Herein, if the conversion table has beenupdated, new axis information is also supplied to the OSD processingunit 182.

[0225] If the OSD processing unit 182 receives the new axis informationfrom the volume value converter 191, the OSD processing unit 182produces, in step S27, display image data associated with the new volumeaxes, corresponding to the adjustment screen 71 described earlier withreference to FIGS. 6 and 7. The OSD processing unit 182 then producesdata corresponding to the icon 72 displayed at a location correspondingto the volume values S′ and Z′ in the current volume axis space shown inFIG. 20 (corresponding to FIG. 18) from the volume values S′ and Z′ inthe initial volume axis space shown in FIG. 19 (corresponding to FIG.17). The OSD processing unit 182 outputs the OSD data corresponding tothe adjustment screen 71 and the icon 72 to the mixer 17 to display iton the display 18.

[0226] In step S28, the system controller 12 determines whether thevolume control operation by the user has been ended, on the basis of asignal received via the signal receiver 11. If it is determined in stepS28 that the volume control operation is not ended, the process returnsto step S25 to repeat step S25 and following steps.

[0227] If it is determined in step S28 that the volume control operationhas been ended, the process proceeds to step S29. In step S29, thesystem controller 12 output a control signal to the history informationmemory 50 to notify that the volume control operation has been ended,and the system controller 12 transfers parameters S and Z indicatingfinal adjustment values associated with image quality to the volumevalue converter 191.

[0228] In step S30, the volume value converter 191 calculates the volumevalues S′ and Z′ using the parameters S and Z indicating the finaladjustment values received from the system controller 12 in accordancewith the conversion table currently stored in the conversion tablememory 194. In step S31, the calculated volume values S′ and Z′ areoutput to the volume value storing unit 192 and stored therein.Thereafter, the process returns to step S22 to repeat step S22 andfollowing steps.

[0229] Thus, by changing the volume axes in the process described abovewith reference to FIG. 16, it becomes possible for the user to easilyadjust parameters on the parameter setting screen including the newcoordinate axes displayed in the OSD manner.

[0230] For example, when a user adjusts a value along a line L in thevolume axis coordinate space shown in FIG. 17 (FIG. 19), the user has toadjust both parameters in the S axis and Z axis. In contrast, in thecase in which a value is adjusted along a line L in the volume axiscoordinate space shown in FIG. 18 (FIG. 20), it is required to adjustonly one parameter in the Z′ axis while maintaining the parameter in theS′ axis at a fixed value. This makes it possible for the user to quicklymake a precise adjustment.

[0231] Furthermore, when the new volume axes are used, it is needed tostore only values in one volume axis. This allows a reduction in theamount of stored data, compared with the case in which values in twovolume axes are stored.

[0232] In the embodiment described above, two parameters associated withimage quality are adjusted. When there are more parameters to beadjusted, the conversion of volume axes can allow a reduction in thenumber of parameters to be adjusted. Thus, the user can easily make anadjustment by performing a simple control operation. Furthermore, theamount of data stored as history information can be reduced.

[0233] The volume axis conversion may be performed using a method otherthan the principal component analysis using linear approximationemployed in the above embodiment. For example, approximation using ahigh-order curve may be employed, or values may be expressed in VQ codesand a conversion may be expressed using a VQ table. In the case in whichapproximation using a high-order curve is employed, a conversion tablebased on the approximation using the high-order curve is stored in theconversion table memory 194, and volume values S′ and Z′ calculatedusing the conversion table based on the approximation using thehigh-order curve are stored in the volume value storing unit 192. In thecase in which values are expressed in VQ codes and the conversion isexpressed using a VQ table, the VQ table is stored in the conversiontable memory 194, and VQ codes are stored in the volume value storingunit 192.

[0234] After volume values corresponding to an extracted feature valueof an input image or an extracted feature value associated with anenvironmental condition are calculated as is performed in the televisionset 1 described earlier with reference to FIG. 1, the calculated volumevalues may be stored, as in the television set 171 described earlierwith reference to FIG. 14, so that new volume axes can be calculatedfrom the stored volume values.

[0235] In a case in which stored data expressed in two volume axes canbe classified into specific patterns on the basis of temporal analysisor feature values of images or environmental parameters, if, instead ofstoring data expressed not in two volume axes, the patterns areexpressed in a VQ table and data are expressed in VQ codes, then thedata size of stored history information can be reduced.

[0236] In the television set 1 or the television set 171, if the historyinformation memory 50 or a circuit board including the historyinformation memory 181 (for example, the image signal processing unit 15or the image signal processing unit 175) is constructed in the form of aremovable unit, it becomes possible to upgrade the television set 1 orthe television set 171 by replacing the unit.

[0237] If history information indicating preferences associated withimages is collected from a great number of users, the collected historyinformation is very useful in designing parameters associated imagequality of new television sets or other display devices. That is, if thehistory information memory 50 or a circuit board including the historyinformation memory 181 is constructed in the form of a removable unit,it becomes possible to collect stored history information for use indesigning parameters to achieve better image quality.

[0238] In the image signal processing unit 15 described above withreference to FIG. 1 or in the image signal processing unit 175 describedabove with reference to FIG. 14, a linear expression is used as theprediction equation in generating HD signals. The prediction equationused in generating HD signals is not limited to linear expressions buthigh-order expressions may also be employed.

[0239] The television set 1 shown in FIG. 1 and the television set 171shown in FIG. 14 may include, or may be connectable to, arecording/playing back apparatus for recording/reproducing content dataonto/from recording medium (not shown) such as a magnetic tape, anoptical disk, a magnetic disk, a magnetooptical disk, or a semiconductormemory.

[0240] This makes it possible for the television set 1 or the televisionset 171 to record a HD signal converted from received SD-signalbroadcast data on storage medium and/or read SD-signal video data from astorage medium and convert it into a HD signal to play back it or torecord it on another storage medium. That is, the present invention canbe applied not only to broadcast data but any type of content data.

[0241] In the image signal processing unit 15 or the image signalprocessing unit 175 described above, a SD signal (525i signal) isconverted into a HD signal (1050i signal). However, in the presentinvention, the conversion is not limited to that from a SD signal to aHD signal. The present invention may also be advantageously used when afirst video signal is converted into a second video signal using aprediction equation.

[0242] Furthermore, in the image signal processing unit 15 or the imagesignal processing unit 175 described above, the information signal is avideo signal. However, in the present invention, the information signalis not limited to video signals. The present invention may also beadvantageously employed when the information signal is an audio signal.

[0243] In the coefficient seed data generator 121 described above withreference to FIG. 12, the SD signal generator 143 produces a SD signalused as a student signal from a HD signal used as a teacher signal, andlearning is performed. A HD signal used as a teacher signal and a SDsignal used as a student signal may be simultaneously acquired using animaging device or the like and learning may be performed using those HDsignal and SD signal acquired separately.

[0244] In the history information memory 50 of the image signalprocessing unit 15 or in the history information memory 181 of the imagesignal processing unit 175 described above, history informationassociated with parameters are stored (more specifically, the historyinformation is stored in the volume value generator 64 of the historyinformation memory 50 or in the volume value storing unit 192 of thehistory information memory 181), and coefficient seed data w10 to wn9are stored in the information memory bank 51. The history informationmemory 50 or the history information memory 181 may further storeinformation associated with other items.

[0245] The process described above may be executed by software. When theprocess is executed by software, a program forming the software may beinstalled from a storage medium onto a computer which is provided asdedicated hardware or may be installed onto a general-purpose computercapable of performing various processes in accordance with variousprograms installed thereon.

[0246] Various types of storage media such as those shown in FIG. 1 or14 may be used for the above purpose. That is, specific examples ofstorage media for this purpose include a magnetic disk 21 (such as afloppy disk), an optical disk 22 (such as a CD-ROM (Compact Disk-ReadOnly Memory) and a DVD (Digital Versatile Disk)), a magnetooptical disk23 (such as a MD (Mini-Disk)), and a semiconductor memory 24, in theform of a package medium on which a program is stored and which issupplied to a user separately from a computer.

[0247] In the present description, the steps described in the programstored in the storage medium may be performed either in time sequence inaccordance with the order described in the program or in a parallel orseparate fashion.

[0248] As can be understood from the above-description, the presentinvention is useful in processing content data.

[0249] In particular, the present invention makes it possible to processcontent data in a manner really intended by a user.

[0250] Furthermore, the present invention makes it possible for a userto easily and quickly make an adjustment. The parameters used in theadjustment can be expressed in simple forms and the parameters can bestored in a compressed form.

What is claimed is:
 1. An information processing apparatus forprocessing content data, comprising: processing means for processing thecontent data; acquisition means for acquiring first information forcontrolling the processing means; and generation means for generatingsecond information using a value obtained by weighting the firstinformation acquired by the acquisition means, wherein the processingmeans processes the content data on the basis of the second informationgenerated by the generation means.
 2. An information processingapparatus according to claim 1, further comprising input means forreceiving a command/data issued by a user wherein the acquisition meansacquires, as the first information, an adjustment value input by theuser via the input means; and the processing means processes the contentdata such that when an automatic adjustment command is input by the uservia the input means, the processing means processes the content data onthe basis of the second information generated by the generation means,while in the case in which the automatic adjustment command is notissued by the user via the input means, when the adjustment value isinput by the user via the input means, the processing means processesthe content data on the basis of the first information acquired by theacquisition means.
 3. An information processing apparatus according toclaim 1, wherein the generation means generates the second informationby performing the weighting such that a greatest weight is applied tothe median of the first information.
 4. An information processingapparatus according to claim 1, further comprising: input means operatedby a user to input a control command/data; and control command/datainput detection means for detecting the status of the inputting of thecontrol command/data, wherein the generation means generates the secondinformation from the first information using a weight depending on thestatus of the inputting of the control command/data.
 5. An informationprocessing apparatus according to claim 4, wherein the controlcommand/data input detection means is control operation time measurementmeans for measuring a time spent in the inputting of the controlcommand/data; and the generation means increases the weight withincreasing time spent in the inputting of the control command/data. 6.An information processing apparatus according to claim 1, furthercomprising feature detection means for detecting features of the contentdata, wherein the generation means generates second information for eachfeature detected by the feature detection means for the content data;and the processing means processes the content data using the secondinformation corresponding to a feature of the content data detected bythe feature detection means.
 7. An information processing apparatusaccording to claim 6, wherein the feature detection means detects, as afeature of the content data, the variance of image levels.
 8. Aninformation processing apparatus according to claim 6, wherein thefeature detection means detects, as a feature of the content data, themean image level.
 9. An information processing apparatus according toclaim 1, further comprising environmental information detection meansfor detecting environmental information associated with an environmentalcondition, wherein the generation means generates second information foreach piece of environmental information detected by the environmentalinformation detection means; and the processing means processes thecontent data using second information corresponding to the environmentalinformation detected by the environmental information detection means.10. An information processing apparatus according to claim 9, whereinthe environmental information detection means detects, as theenvironmental information, the temperature in the ambient.
 11. Aninformation processing apparatus according to claim 9, wherein theenvironmental information detection means detects, as the environmentalinformation, the humidity in the ambient.
 12. An information processingapparatus according to claim 9, wherein the environmental informationdetection means detects, as the environmental information, thebrightness of a light in the ambient.
 13. An information processingapparatus according to claim 1, further comprising informationextraction means for extracting information associated with the contentdata, wherein the generation means generates second information for eachpiece of information extracted by the information extraction means; andthe processing means processes the content data using second informationcorresponding to the information extracted by the information extractionmeans.
 14. An information processing apparatus according to claim 1,further comprising storage means for storing the second informationgenerated by the generation means.
 15. An information processingapparatus according to claim 14, wherein the storage means is formedsuch that it can be removed from the information processing apparatus.16. An information processing method for an information processingapparatus to process content data, comprising the steps of: processingthe content data; acquiring first information for controlling theprocessing step; and generating second information using a valueobtained by weighting the first information acquired in the acquisitionstep, wherein in the processing step, the content data is processed onthe basis of the second information generated in the generation step.17. A storage medium including a program stored thereon for controllingan information processing apparatus for processing content data, theprogram comprising the steps of: processing the content data; acquiringfirst information for controlling the processing step; and generatingsecond information using a value obtained by weighting the firstinformation acquired in the acquisition step, wherein in the processingstep, the content data is processed on the basis of the secondinformation generated in the generation step.
 18. A computer-executableprogram for controlling an information apparatus for processing contentdata, comprising the steps of: processing the content data; acquiringfirst information for controlling the processing step; and generatingsecond information using a value obtained by weighting the firstinformation acquired in the acquisition step, wherein in the processingstep, the content data is processed on the basis of the secondinformation generated in the generation step.
 19. An informationprocessing apparatus for processing content data, comprising: processingmeans for processing the content data; acquisition means for acquiringfirst information and second information for controlling the processingmeans; detection means for detecting a relationship between the firstinformation and the second information acquired by the acquisitionmeans; and generation means for generating third information and fourthinformation by converting the first information and the secondinformation in accordance with the relationship detected by thedetection means; wherein the processing means processes the content datain accordance with the relationship detected by the detection means andthe third information and fourth information generated by the generationmeans.
 20. An information processing apparatus according to claim 19,wherein the detection means detects the relationship between the firstinformation and the second information, by using a linear expression.21. An information processing apparatus according to claim 19, whereinthe detection means detects the relationship between the firstinformation and the second information, by using a high-orderexpression.
 22. An information processing apparatus according to claim19, wherein the detection means detects the relationship between thefirst information and the second information, by using a vectorquantization table and vector quantization codes.
 23. An informationprocessing apparatus according to claim 19, wherein the detection meanscalculates coordinate axes on the basis of the detected relationshipbetween the first information and the second information, and thedetection means produces a conversion table used to generate the thirdinformation and the fourth information by converting the firstinformation and the second information, respectively; and the generationmeans generates the third information and the fourth information byconverting the first information and the second information on the basisof the conversion table generated by the detection means.
 24. Aninformation processing apparatus according to claim 23, furthercomprising display control means for controlling displaying ofinformation other than the content data, wherein the display controlmeans controls displaying of coordinates of the third information andthe fourth information generated by the generation means along thecoordinate axes calculated by the detection means.
 25. An informationprocessing apparatus according to claim 23, further comprising storagemeans for storing the conversion table generated by the detection means.26. An information processing apparatus according to claim 19, furthercomprising storage means for storing the third information and thefourth information generated by the generation means.
 27. An informationprocessing apparatus according to claim 26, wherein when a greaternumber of pieces of third information and fourth information than apredetermined number are stored in the storage means, the detectionmeans detects the relationship between the first and the secondinformation.
 28. An information processing apparatus according to claim26, wherein the storage means is formed such that it can be removed fromthe information processing apparatus.
 29. An information processingapparatus according to claim 23, further comprising input means forreceiving a command/data issued by a user, wherein in response toreceiving via the input means a command to generate the coordinate axes,the detection means detects the relationship between the firstinformation and the second information and generates the axes.
 30. Aninformation processing method for an information processing apparatus toprocess content data, comprising the steps of: processing the contentdata; acquiring first information and second information for controllingthe processing step; detecting a relationship between the firstinformation and the second information acquired in the acquisition step;and generating third information and fourth information by convertingthe first information and the second information in accordance with therelationship detected in the detection step; wherein in the processingstep, the content data is processed in accordance with the relationshipdetected in the detection step and the third information and fourthinformation generated in the generation step.
 31. A storage mediumincluding a program stored thereon for controlling an informationprocessing apparatus for processing content data, the program comprisingthe steps of: processing the content data; acquiring first informationand second information for controlling the processing step; detecting arelationship between the first information and the second informationacquired in the acquisition step; and generating third information andfourth information by converting the first information and the secondinformation in accordance with the relationship detected in thedetection step; wherein in the processing step, the content data isprocessed in accordance with the relationship detected in the detectionstep and the third information and fourth information generated in thegeneration step.
 32. A computer-executable program for controlling aninformation apparatus for processing content data, comprising the stepsof: processing the content data; acquiring first information and secondinformation for controlling the processing step; detecting arelationship between the first information and the second informationacquired in the acquisition step; and generating third information andfourth information by converting the first information and the secondinformation in accordance with the relationship detected in thedetection step; wherein in the processing step, the content data isprocessed in accordance with the relationship detected in the detectionstep and the third information and fourth information generated in thegeneration step.